In recent times, the field of agriculture has been in urgent need of modernizing, since the amount of manual work people need to put in to check if plants are growing correctly is still highly extensive. Despite several advances in agricultural technology, people working in the agricultural industry still need to have the ability to sort and recognize different plants and weeds, which takes a lot of time and effort in the long term. The potential is ripe for this trillion-dollar industry to be greatly impacted by technological innovations that cut down on the requirement for manual labor, and this is where Artificial Intelligence can actually benefit the workers in this field, as the time and energy required to identify plant seedlings will be greatly shortened by the use of AI and Deep Learning. The ability to do so far more efficiently and even more effectively than experienced manual labor, could lead to better crop yields, the freeing up of human inolvement for higher-order agricultural decision making, and in the long term will result in more sustainable environmental practices in agriculture as well.
The aim of this project is to Build a Convolutional Neural Netowrk to classify plant seedlings into their respective categories.
The Aarhus University Signal Processing group, in collaboration with the University of Southern Denmark, has recently released a dataset containing images of unique plants belonging to 12 different species.
Due to the large volume of data, the images were converted to the images.npy file and the labels are also put into Labels.csv, so that you can work on the data/project seamlessly without having to worry about the high data volume.
The goal of the project is to create a classifier capable of determining a plant's species from an image.
List of Species
This is a commented Python Notebook file in which all the instructions and tasks to be performed are mentioned.
Blanks '___' are provided in the notebook that need to be filled with an appropriate code to get the correct result
With every '___' blank, there is a comment that briefly describes what needs to be filled in the blank space
Identify the task to be performed correctly and only then proceed to write the required code
Fill the code wherever asked by the commented lines like "# write your code here" or "# complete the code"
Running incomplete code may throw an error
Please run the codes in a sequential manner from the beginning to avoid any unnecessary errors
Add the results/observations derived from the analysis in the presentation and submit the same in .pdf format
# Installing the libraries with the specified version.
# uncomment and run the following line if Google Colab is being used
!pip install tensorflow==2.15.0 scikit-learn==1.2.2 seaborn==0.13.1 matplotlib==3.7.1 numpy==1.25.2 pandas==1.5.3 opencv-python==4.8.0.76 -q --user
# Installing the libraries with the specified version.
# uncomment and run the following lines if Jupyter Notebook is being used
#!pip install tensorflow==2.13.0 scikit-learn==1.2.2 seaborn==0.11.1 matplotlib==3.3.4 numpy==1.24.3 pandas==1.5.2 opencv-python==4.8.0.76 -q --user
Note: After running the above cell, kindly restart the notebook kernel and run all cells sequentially from the start again.
import os
import numpy as np # Importing numpy for Matrix Operations
import pandas as pd # Importing pandas to read CSV files
import matplotlib.pyplot as plt # Importting matplotlib for Plotting and visualizing images
import math # Importing math module to perform mathematical operations
import cv2 # Importing openCV for image processing
import seaborn as sns # Importing seaborn to plot graphs
# Tensorflow modules
import tensorflow as tf
from tensorflow.keras.preprocessing.image import ImageDataGenerator # Importing the ImageDataGenerator for data augmentation
from tensorflow.keras.models import Sequential # Importing the sequential module to define a sequential model
from tensorflow.keras.layers import Dense,Dropout,Flatten,Conv2D,MaxPooling2D,BatchNormalization # Defining all the layers to build our CNN Model
from tensorflow.keras.optimizers import Adam,SGD # Importing the optimizers which can be used in our model
from sklearn import preprocessing # Importing the preprocessing module to preprocess the data
from sklearn.model_selection import train_test_split # Importing train_test_split function to split the data into train and test
from sklearn.metrics import confusion_matrix # Importing confusion_matrix to plot the confusion matrix
from sklearn.preprocessing import LabelBinarizer
# Display images using OpenCV
from google.colab.patches import cv2_imshow # Importing cv2_imshow from google.patches to display images
from sklearn.model_selection import train_test_split
from tensorflow.keras import backend
from keras.callbacks import ReduceLROnPlateau
import random
# Ignore warnings
import warnings
warnings.filterwarnings('ignore')
# Uncomment and run the below code if you are using google colab
from google.colab import drive
drive.mount('/content/drive')
Mounted at /content/drive
# Load the image file of dataset
images = np.load('/content/drive/MyDrive/Colab Datafiles/P5_Dataset/images.npy') # Complete the code to read the dataset
# Load the labels file of dataset
labels = pd.read_csv('/content/drive/MyDrive/Colab Datafiles/P5_Dataset/Labels.csv') # Complete the code to read the dataset
print(images.shape) # Complete the code to check the shape
print(labels.shape) # Complete the code to check the shape
(4750, 128, 128, 3) (4750, 1)
def plot_images(images,labels):
num_classes=10 # Number of Classes
categories=np.unique(labels)
keys=dict(labels['Label']) # Obtaing the unique classes from y_train
rows = 3 # Defining number of rows=3
cols = 4 # Defining number of columns=4
fig = plt.figure(figsize=(10, 8)) # Defining the figure size to 10x8
for i in range(cols):
for j in range(rows):
random_index = np.random.randint(0, len(labels)) # Generating random indices from the data and plotting the images
ax = fig.add_subplot(rows, cols, i * rows + j + 1) # Adding subplots with 3 rows and 4 columns
ax.imshow(images[random_index, :]) # Plotting the image
ax.set_title(keys[random_index])
plt.show()
plot_images(images,labels) # Complete the code to input the images and labels to the function and plot the images with their labels
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
def plot_images(images, labels):
num_classes = 10 # Number of Classes
categories = np.unique(labels['Label'])
keys = dict(enumerate(categories)) # Obtain the unique classes from y_train
rows = 3 # Define number of rows
cols = 4 # Define number of columns
fig = plt.figure(figsize=(10, 8)) # Define the figure size
for i in range(cols):
for j in range(rows):
random_index = np.random.randint(0, len(labels)) # Generate random indices from the data and plot the images
ax = fig.add_subplot(rows, cols, i * rows + j + 1) # Add subplots with 3 rows and 4 columns
ax.imshow(images[random_index]) # Plot the image
ax.set_title(labels.iloc[random_index]['Label']) # Set the title as the label
plt.show()
plot_images(images, labels)
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
# Plot one image of each plant to check if they are labeled properly
def plot_unique_images(images, labels):
unique_labels = labels['Label'].unique()
num_classes = len(unique_labels)
fig = plt.figure(figsize=(15, 10)) # Define the figure size
for i, label in enumerate(unique_labels):
# Get the first index of the current label
index = labels[labels['Label'] == label].index[0]
ax = fig.add_subplot(3, 4, i + 1) # Add subplots (3 rows, 4 columns)
ax.imshow(images[index]) # Plot the image
ax.set_title(label) # Set the title as the label
ax.axis('off') # Hide the axes
plt.tight_layout() # Adjust the layout to fit everything
plt.show()
plot_unique_images(images, labels)
# Get the count of each plant type
label_counts = labels['Label'].value_counts()
# Print the counts
print(label_counts)
Label Loose Silky-bent 654 Common Chickweed 611 Scentless Mayweed 516 Small-flowered Cranesbill 496 Fat Hen 475 Charlock 390 Sugar beet 385 Cleavers 287 Black-grass 263 Shepherds Purse 231 Common wheat 221 Maize 221 Name: count, dtype: int64
Image Visualization
Count Plot for Each Category
Next Steps
sns.countplot(y=labels['Label']) # Complete the code to check for data imbalance
plt.xticks(rotation='vertical')
plt.show()
# Sort the labels in ascending order
sorted_labels = labels['Label'].value_counts().sort_values().index
# Plot the countplot with sorted labels
sns.countplot(y=labels['Label'], order=sorted_labels)
plt.xticks(rotation='vertical')
plt.show()
# Converting the images from BGR to RGB using cvtColor function of OpenCV
for i in range(len(images)):
images[i] = cv2.cvtColor(images[i], cv2.COLOR_BGR2RGB) # Complete the code to convert the images from BGR to RGB
As the size of the images is large, it may be computationally expensive to train on these larger images; therefore, it is preferable to reduce the image size from 128 to 64.
images_decreased=[]
height = 64 # Complete the code to define the height as 64
width = 64 # Complete the code to define the width as 64
dimensions = (width, height)
for i in range(len(images)):
images_decreased.append( cv2.resize(images[i], dimensions, interpolation=cv2.INTER_LINEAR))
Image before resizing
plt.imshow(images[3])
<matplotlib.image.AxesImage at 0x7b34b2ce9f30>
Image after resizing
plt.imshow(images_decreased[3])
<matplotlib.image.AxesImage at 0x7b34b6c742e0>
# Split the data into temporary (80%) and test (20%) sets
X_temp, X_test, y_temp, y_test = train_test_split(
np.array(images_decreased),
labels['Label'],
test_size=0.1,
random_state=42,
stratify=labels['Label']
)
# Complete the code to split the data with test_size as 0.1
# Split the temporary set into training (80% of 90%) and validation (20% of 90%) sets
X_train, X_val, y_train, y_val = train_test_split(
X_temp,
y_temp,
test_size=0.1,
random_state=42,
stratify=y_temp
)
# Complete the code to split the data with test_size as 0.1
# Complete the code to check the shape of train, validation and test data
print(X_train.shape, y_train.shape)
print(X_val.shape, y_val.shape)
print(X_test.shape, y_test.shape)
(3847, 64, 64, 3) (3847,) (428, 64, 64, 3) (428,) (475, 64, 64, 3) (475,)
# Convert labels from names to one hot vectors.
# We have already used encoding methods like onehotencoder and labelencoder earlier so now we will be using a new encoding method called labelBinarizer.
# Labelbinarizer works similar to onehotencoder
from sklearn.preprocessing import LabelBinarizer
# Initialize the LabelBinarizer
enc = LabelBinarizer()
# Fit and transform y_train
y_train_encoded = enc.fit_transform(y_train)
# Transform y_val
y_val_encoded = enc.transform(y_val)
# Transform y_test
y_test_encoded = enc.transform(y_test) # Complete the code to transform y_test
y_train_encoded.shape, y_val_encoded.shape, y_test_encoded.shape # Complete the code to check the shape of train, validation, and test data
((3847, 12), (428, 12), (475, 12))
Since the image pixel values range from 0-255, our method of normalization here will be scaling - we shall divide all the pixel values by 255 to standardize the images to have values between 0-1.
# Normalize the image pixels
X_train_normalized = X_train.astype('float32') / 255.0
X_val_normalized = X_val.astype('float32') / 255.0
X_test_normalized = X_test.astype('float32') / 255.0
# Clearing backend
backend.clear_session()
# Fixing the seed for random number generators
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)
# Initializing a sequential model
model1 = Sequential()
# Adding the first conv layer with 128 filters and kernel size 3x3, padding 'same' provides the output size same as the input size
# Input_shape denotes input image dimension of images
model1.add(Conv2D(128, (3, 3), activation='relu', padding="same", input_shape=(64, 64, 3)))
# Adding the max pooling to reduce the size of output of first conv layer
model1.add(MaxPooling2D((2, 2), padding='same'))
# Creating two similar convolution and max-pooling layers with activation = relu
model1.add(Conv2D(64, (3, 3), activation='relu', padding="same"))
model1.add(MaxPooling2D((2, 2), padding='same'))
model1.add(Conv2D(32, (3, 3), activation='relu', padding="same"))
model1.add(MaxPooling2D((2, 2), padding='same'))
# Flattening the output of the conv layer after max pooling to make it ready for creating dense connections
model1.add(Flatten())
# Adding a fully connected dense layer with 16 neurons
model1.add(Dense(16, activation='relu'))
model1.add(Dropout(0.3))
# Adding the output layer with 12 neurons and activation function as softmax since this is a multi-class classification problem
model1.add(Dense(12, activation='softmax'))
# Using the Adam Optimizer
opt = Adam()
# Compiling the model using suitable metric for loss function
model1.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
# Generating the summary of the model
model1.summary()
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 64, 64, 128) 3584
max_pooling2d (MaxPooling2 (None, 32, 32, 128) 0
D)
conv2d_1 (Conv2D) (None, 32, 32, 64) 73792
max_pooling2d_1 (MaxPoolin (None, 16, 16, 64) 0
g2D)
conv2d_2 (Conv2D) (None, 16, 16, 32) 18464
max_pooling2d_2 (MaxPoolin (None, 8, 8, 32) 0
g2D)
flatten (Flatten) (None, 2048) 0
dense (Dense) (None, 16) 32784
dropout (Dropout) (None, 16) 0
dense_1 (Dense) (None, 12) 204
=================================================================
Total params: 128828 (503.23 KB)
Trainable params: 128828 (503.23 KB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
Fitting the model on the train data
# Fitting the model on the train data
history_1 = model1.fit(
X_train_normalized, y_train_encoded,
epochs=30,
validation_data=(X_val_normalized, y_val_encoded),
batch_size=32,
verbose=2
)
Epoch 1/30 121/121 - 3s - loss: 2.4545 - accuracy: 0.1050 - val_loss: 2.4389 - val_accuracy: 0.1379 - 3s/epoch - 23ms/step Epoch 2/30 121/121 - 1s - loss: 2.4338 - accuracy: 0.1362 - val_loss: 2.4162 - val_accuracy: 0.2944 - 1s/epoch - 11ms/step Epoch 3/30 121/121 - 1s - loss: 2.1751 - accuracy: 0.2797 - val_loss: 1.8992 - val_accuracy: 0.3879 - 1s/epoch - 11ms/step Epoch 4/30 121/121 - 1s - loss: 1.9310 - accuracy: 0.3488 - val_loss: 1.7489 - val_accuracy: 0.4206 - 1s/epoch - 10ms/step Epoch 5/30 121/121 - 1s - loss: 1.7789 - accuracy: 0.3915 - val_loss: 1.5470 - val_accuracy: 0.4953 - 1s/epoch - 9ms/step Epoch 6/30 121/121 - 1s - loss: 1.6323 - accuracy: 0.4333 - val_loss: 1.4522 - val_accuracy: 0.5421 - 1s/epoch - 9ms/step Epoch 7/30 121/121 - 1s - loss: 1.5449 - accuracy: 0.4643 - val_loss: 1.2875 - val_accuracy: 0.5678 - 1s/epoch - 9ms/step Epoch 8/30 121/121 - 1s - loss: 1.4362 - accuracy: 0.5012 - val_loss: 1.1836 - val_accuracy: 0.6075 - 1s/epoch - 9ms/step Epoch 9/30 121/121 - 1s - loss: 1.3593 - accuracy: 0.5253 - val_loss: 1.1559 - val_accuracy: 0.5958 - 1s/epoch - 9ms/step Epoch 10/30 121/121 - 1s - loss: 1.2987 - accuracy: 0.5344 - val_loss: 1.1702 - val_accuracy: 0.6121 - 1s/epoch - 9ms/step Epoch 11/30 121/121 - 1s - loss: 1.2449 - accuracy: 0.5586 - val_loss: 1.0647 - val_accuracy: 0.6402 - 1s/epoch - 9ms/step Epoch 12/30 121/121 - 1s - loss: 1.2055 - accuracy: 0.5716 - val_loss: 1.0547 - val_accuracy: 0.6472 - 1s/epoch - 9ms/step Epoch 13/30 121/121 - 1s - loss: 1.1836 - accuracy: 0.5797 - val_loss: 1.0610 - val_accuracy: 0.6472 - 1s/epoch - 11ms/step Epoch 14/30 121/121 - 1s - loss: 1.1339 - accuracy: 0.5963 - val_loss: 1.0521 - val_accuracy: 0.6379 - 1s/epoch - 10ms/step Epoch 15/30 121/121 - 1s - loss: 1.1288 - accuracy: 0.6010 - val_loss: 1.0210 - val_accuracy: 0.6822 - 1s/epoch - 11ms/step Epoch 16/30 121/121 - 1s - loss: 1.0931 - accuracy: 0.6028 - val_loss: 1.0390 - val_accuracy: 0.6449 - 1s/epoch - 9ms/step Epoch 17/30 121/121 - 1s - loss: 1.0669 - accuracy: 0.6041 - val_loss: 1.0066 - val_accuracy: 0.6752 - 1s/epoch - 9ms/step Epoch 18/30 121/121 - 1s - loss: 1.0266 - accuracy: 0.6272 - val_loss: 1.0090 - val_accuracy: 0.6776 - 1s/epoch - 9ms/step Epoch 19/30 121/121 - 1s - loss: 1.0368 - accuracy: 0.6163 - val_loss: 1.0484 - val_accuracy: 0.6776 - 1s/epoch - 9ms/step Epoch 20/30 121/121 - 1s - loss: 1.0089 - accuracy: 0.6298 - val_loss: 1.0436 - val_accuracy: 0.6682 - 1s/epoch - 9ms/step Epoch 21/30 121/121 - 1s - loss: 0.9833 - accuracy: 0.6361 - val_loss: 1.0071 - val_accuracy: 0.6706 - 1s/epoch - 9ms/step Epoch 22/30 121/121 - 1s - loss: 0.9963 - accuracy: 0.6324 - val_loss: 0.9904 - val_accuracy: 0.6893 - 1s/epoch - 9ms/step Epoch 23/30 121/121 - 1s - loss: 0.9689 - accuracy: 0.6405 - val_loss: 1.0544 - val_accuracy: 0.6659 - 1s/epoch - 9ms/step Epoch 24/30 121/121 - 1s - loss: 0.9400 - accuracy: 0.6538 - val_loss: 1.0713 - val_accuracy: 0.6822 - 1s/epoch - 10ms/step Epoch 25/30 121/121 - 1s - loss: 0.9237 - accuracy: 0.6543 - val_loss: 1.0022 - val_accuracy: 0.6893 - 1s/epoch - 11ms/step Epoch 26/30 121/121 - 1s - loss: 0.8718 - accuracy: 0.6769 - val_loss: 0.9811 - val_accuracy: 0.6939 - 1s/epoch - 11ms/step Epoch 27/30 121/121 - 1s - loss: 0.9001 - accuracy: 0.6668 - val_loss: 1.0194 - val_accuracy: 0.6939 - 1s/epoch - 10ms/step Epoch 28/30 121/121 - 1s - loss: 0.8779 - accuracy: 0.6769 - val_loss: 1.0069 - val_accuracy: 0.6822 - 1s/epoch - 9ms/step Epoch 29/30 121/121 - 1s - loss: 0.8575 - accuracy: 0.6803 - val_loss: 0.9688 - val_accuracy: 0.6963 - 1s/epoch - 9ms/step Epoch 30/30 121/121 - 1s - loss: 0.8629 - accuracy: 0.6849 - val_loss: 0.9994 - val_accuracy: 0.7033 - 1s/epoch - 9ms/step
Model Evaluation
plt.plot(history_1.history['accuracy'])
plt.plot(history_1.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
Evaluate the model on test data
# Complete the code to evaluate the model on test data
accuracy = model1.evaluate(X_test_normalized, y_test_encoded, verbose=2)
print(f'Test accuracy: {accuracy[1]}')
15/15 - 0s - loss: 0.9131 - accuracy: 0.7116 - 83ms/epoch - 6ms/step Test accuracy: 0.7115789651870728
15/15 - 0s - loss: 0.9790 - accuracy: 0.7032 - 82ms/epoch - 5ms/step Test accuracy: 0.70315790176391
Plotting the Confusion Matrix
# Here we would get the output as probablities for each category
y_pred = model1.predict(X_test_normalized) # Complete the code to predict the output probabilities
15/15 [==============================] - 0s 4ms/step
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)
# Plotting the Confusion Matrix using confusion matrix() function which is also predefined in tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg, y_pred_arg) # Complete the code to plot the confusion matrix
f, ax = plt.subplots(figsize=(12, 12))
sns.heatmap(
confusion_matrix,
annot=True,
linewidths=.4,
fmt="d",
square=True,
ax=ax
)
# Setting the labels to both the axes
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels');
ax.set_title('Confusion Matrix');
ax.xaxis.set_ticklabels(list(enc.classes_),rotation=40)
ax.yaxis.set_ticklabels(list(enc.classes_),rotation=20)
plt.show()
Plotting Classification Report
from sklearn import metrics
# Plotting the classification report
cr = metrics.classification_report(y_test_arg, y_pred_arg,target_names=enc.classes_)
print(cr)
precision recall f1-score support
Black-grass 0.00 0.00 0.00 26
Charlock 0.69 0.85 0.76 39
Cleavers 0.71 0.59 0.64 29
Common Chickweed 0.86 0.89 0.87 61
Common wheat 0.33 0.09 0.14 22
Fat Hen 0.81 0.79 0.80 48
Loose Silky-bent 0.57 0.91 0.70 65
Maize 0.65 0.50 0.56 22
Scentless Mayweed 0.64 0.79 0.71 52
Shepherds Purse 0.73 0.48 0.58 23
Small-flowered Cranesbill 0.93 0.80 0.86 50
Sugar beet 0.71 0.84 0.77 38
accuracy 0.71 475
macro avg 0.64 0.63 0.62 475
weighted avg 0.68 0.71 0.68 475
Overall Accuracy:
Class-wise Performance:
Imbalance Impact:
Confusion Matrix Insights:
Macro and Weighted Averages:
Reducing the Learning Rate:
ReduceLRonPlateau() is a function that will be used to decrease the learning rate by some factor, if the loss is not decreasing for some time. This may start decreasing the loss at a smaller learning rate. There is a possibility that the loss may still not decrease. This may lead to executing the learning rate reduction again in an attempt to achieve a lower loss.
# Code to monitor val_accuracy
learning_rate_reduction = ReduceLROnPlateau(monitor='val_accuracy',
patience=3,
verbose=1,
factor=0.5,
min_lr=0.00001)
# Clearing backend
from tensorflow.keras import backend
backend.clear_session()
# Fixing the seed for random number generators
import random
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)
# Complete the code to set the rotation_range to 20
train_datagen = ImageDataGenerator(
rotation_range=20,
fill_mode='nearest'
)
# Take a random sample from the training data to visualize augmentation
sample_index = np.random.randint(0, len(X_train_normalized))
sample_image = X_train_normalized[sample_index]
sample_label = y_train_encoded[sample_index]
# Ensure sample_label is a numpy array
sample_label = np.array(sample_label)
# Generate batches of augmented images
augmented_images = train_datagen.flow(
sample_image.reshape(1, 64, 64, 3),
batch_size=1
)
# Plotting original and augmented images
plt.figure(figsize=(12, 6))
# Original image
plt.subplot(1, 5, 1)
plt.imshow(sample_image)
plt.title(f'Original\nLabel: {enc.inverse_transform(sample_label.reshape(1, -1))[0]}')
plt.axis('off')
# Augmented images
for i in range(4):
aug_img = augmented_images.next()[0]
plt.subplot(1, 5, i + 2)
plt.imshow(aug_img)
plt.title(f'Augmented\nLabel: {enc.inverse_transform(sample_label.reshape(1, -1))[0]}')
plt.axis('off')
plt.tight_layout()
plt.show()
import numpy as np
import matplotlib.pyplot as plt
from tensorflow.keras.preprocessing.image import ImageDataGenerator
# Creating the data generator with augmentation
train_datagen = ImageDataGenerator(
rotation_range=180,
fill_mode='nearest'
)
# Find indices of all 'Maize' samples
maize_indices = np.where(np.argmax(y_train_encoded, axis=1) == np.argmax(enc.transform(['Maize'])))[0]
# Take a random sample from the 'Maize' class
sample_index = np.random.choice(maize_indices)
sample_image = X_train_normalized[sample_index]
sample_label = y_train_encoded[sample_index]
# Ensure sample_label is a numpy array
sample_label = np.array(sample_label)
# Generate batches of augmented images
augmented_images = train_datagen.flow(
sample_image.reshape(1, 64, 64, 3),
batch_size=1
)
# Plotting original and augmented images
plt.figure(figsize=(12, 6))
# Original image
plt.subplot(1, 5, 1)
plt.imshow(sample_image)
plt.title(f'Original\nLabel: Maize')
plt.axis('off')
# Augmented images
for i in range(4):
aug_img = augmented_images.next()[0]
plt.subplot(1, 5, i + 2)
plt.imshow(aug_img)
plt.title(f'Augmented\nLabel: Maize')
plt.axis('off')
plt.tight_layout()
plt.show()
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Conv2D, MaxPooling2D, Flatten, Dense, Dropout, BatchNormalization
from tensorflow.keras.optimizers import Adam
# Initializing a sequential model
model2 = Sequential()
# Adding the first conv layer with 64 filters and kernel size 3x3, padding 'same' provides the output size same as the input size
# Input_shape denotes input image dimension of images
model2.add(Conv2D(64, (3, 3), activation='relu', padding="same", input_shape=(64, 64, 3)))
# Adding max pooling to reduce the size of output of first conv layer
model2.add(MaxPooling2D((2, 2), padding='same'))
# Adding additional convolutional and max-pooling layers
model2.add(Conv2D(32, (3, 3), activation='relu', padding="same"))
model2.add(MaxPooling2D((2, 2), padding='same'))
model2.add(BatchNormalization())
# Flattening the output of the conv layer after max pooling to make it ready for creating dense connections
model2.add(Flatten())
# Adding a fully connected dense layer with 16 neurons
model2.add(Dense(16, activation='relu'))
# Adding dropout with dropout_rate=0.3
model2.add(Dropout(0.3))
# Adding the output layer with 12 neurons and activation functions as softmax since this is a multi-class classification problem
model2.add(Dense(12, activation='softmax'))
# Initializing Adam Optimizer
opt = Adam()
# Compiling the model
model2.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
# Generating the summary of the model
model2.summary()
Model: "sequential_1"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d_2 (Conv2D) (None, 64, 64, 64) 1792
max_pooling2d_2 (MaxPoolin (None, 32, 32, 64) 0
g2D)
conv2d_3 (Conv2D) (None, 32, 32, 32) 18464
max_pooling2d_3 (MaxPoolin (None, 16, 16, 32) 0
g2D)
batch_normalization_1 (Bat (None, 16, 16, 32) 128
chNormalization)
flatten_1 (Flatten) (None, 8192) 0
dense_2 (Dense) (None, 16) 131088
dropout_1 (Dropout) (None, 16) 0
dense_3 (Dense) (None, 12) 204
=================================================================
Total params: 151676 (592.48 KB)
Trainable params: 151612 (592.23 KB)
Non-trainable params: 64 (256.00 Byte)
_________________________________________________________________
Fitting the model on the train data
# Complete the code to fit the model on train data with batch_size=64 and epochs=30
# Epochs
epochs = 30
# Batch size
batch_size = 64
history = model2.fit(train_datagen.flow(X_train_normalized,y_train_encoded,
batch_size=batch_size,
shuffle=False),
epochs=epochs,
steps_per_epoch=X_train_normalized.shape[0] // batch_size,
validation_data=(X_val_normalized,y_val_encoded),
verbose=1,callbacks=[learning_rate_reduction])
Epoch 1/30 60/60 [==============================] - 6s 73ms/step - loss: 2.3168 - accuracy: 0.2199 - val_loss: 2.4360 - val_accuracy: 0.1706 - lr: 0.0010 Epoch 2/30 60/60 [==============================] - 6s 96ms/step - loss: 1.9356 - accuracy: 0.3521 - val_loss: 2.3918 - val_accuracy: 0.3131 - lr: 0.0010 Epoch 3/30 60/60 [==============================] - 5s 77ms/step - loss: 1.7214 - accuracy: 0.4097 - val_loss: 2.2509 - val_accuracy: 0.2967 - lr: 0.0010 Epoch 4/30 60/60 [==============================] - 5s 83ms/step - loss: 1.5865 - accuracy: 0.4578 - val_loss: 2.0215 - val_accuracy: 0.3388 - lr: 0.0010 Epoch 5/30 60/60 [==============================] - 4s 70ms/step - loss: 1.5016 - accuracy: 0.4830 - val_loss: 2.0113 - val_accuracy: 0.3364 - lr: 0.0010 Epoch 6/30 60/60 [==============================] - 6s 98ms/step - loss: 1.4180 - accuracy: 0.5160 - val_loss: 1.7818 - val_accuracy: 0.5514 - lr: 0.0010 Epoch 7/30 60/60 [==============================] - 4s 69ms/step - loss: 1.3429 - accuracy: 0.5387 - val_loss: 1.6348 - val_accuracy: 0.5023 - lr: 0.0010 Epoch 8/30 60/60 [==============================] - 4s 70ms/step - loss: 1.3105 - accuracy: 0.5435 - val_loss: 1.4296 - val_accuracy: 0.5631 - lr: 0.0010 Epoch 9/30 60/60 [==============================] - 6s 93ms/step - loss: 1.2372 - accuracy: 0.5829 - val_loss: 1.3532 - val_accuracy: 0.6659 - lr: 0.0010 Epoch 10/30 60/60 [==============================] - 5s 85ms/step - loss: 1.2040 - accuracy: 0.5855 - val_loss: 1.5063 - val_accuracy: 0.5047 - lr: 0.0010 Epoch 11/30 60/60 [==============================] - 5s 78ms/step - loss: 1.1771 - accuracy: 0.5969 - val_loss: 1.0924 - val_accuracy: 0.6822 - lr: 0.0010 Epoch 12/30 60/60 [==============================] - 4s 68ms/step - loss: 1.1476 - accuracy: 0.6246 - val_loss: 1.3668 - val_accuracy: 0.5935 - lr: 0.0010 Epoch 13/30 60/60 [==============================] - 7s 109ms/step - loss: 1.1145 - accuracy: 0.6196 - val_loss: 1.0709 - val_accuracy: 0.6425 - lr: 0.0010 Epoch 14/30 60/60 [==============================] - ETA: 0s - loss: 1.0808 - accuracy: 0.6371 Epoch 14: ReduceLROnPlateau reducing learning rate to 0.0005000000237487257. 60/60 [==============================] - 4s 69ms/step - loss: 1.0808 - accuracy: 0.6371 - val_loss: 1.2409 - val_accuracy: 0.6215 - lr: 0.0010 Epoch 15/30 60/60 [==============================] - 6s 98ms/step - loss: 1.0440 - accuracy: 0.6376 - val_loss: 0.7739 - val_accuracy: 0.7430 - lr: 5.0000e-04 Epoch 16/30 60/60 [==============================] - 4s 69ms/step - loss: 1.0168 - accuracy: 0.6460 - val_loss: 0.9102 - val_accuracy: 0.7033 - lr: 5.0000e-04 Epoch 17/30 60/60 [==============================] - 6s 97ms/step - loss: 0.9964 - accuracy: 0.6624 - val_loss: 0.9259 - val_accuracy: 0.7079 - lr: 5.0000e-04 Epoch 18/30 60/60 [==============================] - ETA: 0s - loss: 0.9970 - accuracy: 0.6582 Epoch 18: ReduceLROnPlateau reducing learning rate to 0.0002500000118743628. 60/60 [==============================] - 4s 69ms/step - loss: 0.9970 - accuracy: 0.6582 - val_loss: 1.0695 - val_accuracy: 0.6449 - lr: 5.0000e-04 Epoch 19/30 60/60 [==============================] - 5s 76ms/step - loss: 0.9523 - accuracy: 0.6746 - val_loss: 0.8323 - val_accuracy: 0.7336 - lr: 2.5000e-04 Epoch 20/30 60/60 [==============================] - 6s 91ms/step - loss: 0.9600 - accuracy: 0.6743 - val_loss: 0.9031 - val_accuracy: 0.7079 - lr: 2.5000e-04 Epoch 21/30 60/60 [==============================] - 5s 85ms/step - loss: 0.9551 - accuracy: 0.6749 - val_loss: 0.7430 - val_accuracy: 0.7804 - lr: 2.5000e-04 Epoch 22/30 60/60 [==============================] - 6s 102ms/step - loss: 0.9557 - accuracy: 0.6860 - val_loss: 0.9160 - val_accuracy: 0.7126 - lr: 2.5000e-04 Epoch 23/30 60/60 [==============================] - 4s 69ms/step - loss: 0.9392 - accuracy: 0.6778 - val_loss: 1.1639 - val_accuracy: 0.6215 - lr: 2.5000e-04 Epoch 24/30 60/60 [==============================] - ETA: 0s - loss: 0.9181 - accuracy: 0.6886 Epoch 24: ReduceLROnPlateau reducing learning rate to 0.0001250000059371814. 60/60 [==============================] - 6s 99ms/step - loss: 0.9181 - accuracy: 0.6886 - val_loss: 0.7625 - val_accuracy: 0.7407 - lr: 2.5000e-04 Epoch 25/30 60/60 [==============================] - 4s 69ms/step - loss: 0.9049 - accuracy: 0.6973 - val_loss: 0.7564 - val_accuracy: 0.7500 - lr: 1.2500e-04 Epoch 26/30 60/60 [==============================] - 4s 73ms/step - loss: 0.9019 - accuracy: 0.6928 - val_loss: 0.7228 - val_accuracy: 0.7617 - lr: 1.2500e-04 Epoch 27/30 60/60 [==============================] - ETA: 0s - loss: 0.9135 - accuracy: 0.6854 Epoch 27: ReduceLROnPlateau reducing learning rate to 6.25000029685907e-05. 60/60 [==============================] - 5s 82ms/step - loss: 0.9135 - accuracy: 0.6854 - val_loss: 0.7517 - val_accuracy: 0.7477 - lr: 1.2500e-04 Epoch 28/30 60/60 [==============================] - 4s 70ms/step - loss: 0.9103 - accuracy: 0.6815 - val_loss: 0.6924 - val_accuracy: 0.7710 - lr: 6.2500e-05 Epoch 29/30 60/60 [==============================] - 5s 88ms/step - loss: 0.8960 - accuracy: 0.6952 - val_loss: 0.6675 - val_accuracy: 0.7944 - lr: 6.2500e-05 Epoch 30/30 60/60 [==============================] - 5s 79ms/step - loss: 0.8959 - accuracy: 0.6846 - val_loss: 0.6964 - val_accuracy: 0.7664 - lr: 6.2500e-05
Model Evaluation
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
Evaluate the model on test data
# Evaluate the model on test data
accuracy = model2.evaluate(X_test_normalized, y_test_encoded, verbose=2)
print(f'Test accuracy: {accuracy[1]}')
15/15 - 0s - loss: 0.8073 - accuracy: 0.7663 - 73ms/epoch - 5ms/step Test accuracy: 0.7663158178329468
Plotting the Confusion Matrix
# Complete the code to obtain the output probabilities
y_pred = model2.predict(X_test_normalized)
15/15 [==============================] - 0s 3ms/step
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)
# Plotting the Confusion Matrix using confusion matrix() function which is also predefined in tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg) # Complete the code to obatin the confusion matrix
f, ax = plt.subplots(figsize=(12, 12))
sns.heatmap(
confusion_matrix,
annot=True,
linewidths=.4,
fmt="d",
square=True,
ax=ax
)
# Setting the labels to both the axes
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels');
ax.set_title('Confusion Matrix');
ax.xaxis.set_ticklabels(list(enc.classes_),rotation=40)
ax.yaxis.set_ticklabels(list(enc.classes_),rotation=20)
plt.show()
Plotting Classification Report
# Plotting the classification report
cr = metrics.classification_report(y_test_arg, y_pred_arg,target_names=enc.classes_)
print(cr)
precision recall f1-score support
Black-grass 0.37 0.27 0.31 26
Charlock 0.71 0.92 0.80 39
Cleavers 0.73 0.66 0.69 29
Common Chickweed 0.96 0.90 0.93 61
Common wheat 0.71 0.55 0.62 22
Fat Hen 0.75 0.75 0.75 48
Loose Silky-bent 0.69 0.74 0.71 65
Maize 0.69 0.91 0.78 22
Scentless Mayweed 0.78 0.83 0.80 52
Shepherds Purse 0.79 0.65 0.71 23
Small-flowered Cranesbill 0.89 0.84 0.87 50
Sugar beet 0.84 0.82 0.83 38
accuracy 0.77 475
macro avg 0.74 0.74 0.73 475
weighted avg 0.77 0.77 0.76 475
Model Accuracy and Loss
Confusion Matrix Insights
Classification Report
Key Findings
Model 1:
Model 2:
Conclusion Based on the evaluation metrics and performance insights:
Superior Model: Model 2 Model 2 is superior due to its higher accuracy, improved performance across most classes, and better generalization capabilities. It effectively addresses some of the limitations observed in Model 1, making it a more robust choice for the plant seedling classification task.
Comment on the final model you have selected and use the same in the below code to visualize the image.
# Visualizing the predicted and correct label of images from test data
# Image 2
plt.figure(figsize=(2, 2))
plt.imshow(X_test[2])
plt.show()
# Predict the label for the 2nd image
print('Predicted Label', enc.inverse_transform(model2.predict(X_test_normalized[2].reshape(1, 64, 64, 3))))
print('True Label', enc.inverse_transform(y_test_encoded)[2])
# Image 33
plt.figure(figsize=(2, 2))
plt.imshow(X_test[33])
plt.show()
# Predict the label for the 33rd image
print('Predicted Label', enc.inverse_transform(model2.predict(X_test_normalized[33].reshape(1, 64, 64, 3))))
print('True Label', enc.inverse_transform(y_test_encoded)[33])
# Image 59
plt.figure(figsize=(2, 2))
plt.imshow(X_test[59])
plt.show()
# Predict the label for the 59th image
print('Predicted Label', enc.inverse_transform(model2.predict(X_test_normalized[59].reshape(1, 64, 64, 3))))
print('True Label', enc.inverse_transform(y_test_encoded)[59])
# Image 36
plt.figure(figsize=(2, 2))
plt.imshow(X_test[36])
plt.show()
# Predict the label for the 36th image
print('Predicted Label', enc.inverse_transform(model2.predict(X_test_normalized[36].reshape(1, 64, 64, 3))))
print('True Label', enc.inverse_transform(y_test_encoded)[36])
1/1 [==============================] - 0s 17ms/step Predicted Label ['Small-flowered Cranesbill'] True Label Small-flowered Cranesbill
1/1 [==============================] - 0s 19ms/step Predicted Label ['Cleavers'] True Label Cleavers
1/1 [==============================] - 0s 24ms/step Predicted Label ['Common Chickweed'] True Label Common Chickweed
1/1 [==============================] - 0s 18ms/step Predicted Label ['Shepherds Purse'] True Label Shepherds Purse
Correct Predictions:
The model correctly predicted the labels for all four images tested:
Model Confidence:
Data Augmentation and Learning Rate Reduction Impact:
The improvements in accuracy and correct predictions on test images reflect the positive impact of data augmentation and learning rate reduction on the model's performance.
Class Generalization:
Conclusion The current observations are promising and indicate that the enhancements made to the model (data augmentation and learning rate reduction) have positively impacted its performance. However, further testing with a broader set of test images is recommended to comprehensively evaluate the model's accuracy and robustness across all classes. This will help in identifying any remaining weaknesses and guiding further improvements.
Actionable Insights
Model Performance:
• The model has achieved a test accuracy of approximately 76.4%, showing a significant improvement over the initial model. The use of data augmentation and learning rate reduction has been effective in enhancing model performance.
Class Imbalance:
• Despite the improvements, the model still struggles with underrepresented classes such as 'Black-grass' and 'Common wheat'. Addressing class imbalance through oversampling, undersampling, or class weighting could further improve model accuracy.
Misclassifications:
• Certain classes are more frequently misclassified, indicating that the model may benefit from additional training data or more nuanced features to better distinguish between similar-looking classes.
Data Augmentation:
• The application of data augmentation has shown to be beneficial. Continuing to use and possibly expanding data augmentation techniques (such as varying degrees of rotation, flips, zooms, and shifts) can further improve model robustness.
Model Complexity:
• Increasing the complexity of the model by adding more layers or neurons could help capture more intricate patterns in the data, potentially improving performance on challenging classes.
Regularization Techniques:
• Regularization methods such as dropout layers and L2 regularization should continue to be employed to prevent overfitting and improve generalization to new data.
Business Recommendations
Investment in Data Collection:
• Invest in collecting more data, especially for underrepresented classes like 'Black-grass' and 'Common wheat'. This will help in training a more balanced and accurate model.
Deployment and Monitoring:
• Deploy the current model in a controlled environment to monitor its performance on real-world data. Use this phase to gather more data and identify any additional areas for improvement.
User Feedback Integration:
• Implement a feedback mechanism for users (e.g., farmers, agronomists) to report misclassifications. This feedback can be used to continuously retrain and improve the model.
Educational Resources:
• Provide training and educational resources to users on how to interpret and use the model's predictions effectively. This can help in better adoption and utilization of the technology.
Collaborative Research:
• Collaborate with agricultural research institutions and universities to further refine the model. Sharing insights and data can lead to more robust and accurate models.
Scalability and Infrastructure:
• Ensure that the infrastructure is in place to scale the model for broader use. This includes cloud-based solutions for handling large amounts of data and deploying the model across different regions and environments.
Continuous Improvement:
• Establish a cycle of continuous improvement where the model is regularly updated with new data and retrained to maintain and improve its accuracy and reliability over time.
Integration with Other Technologies:
• Explore integrating the model with other agricultural technologies such as automated farming equipment, drones, and IoT sensors to create a comprehensive smart farming solution. • By following these insights and recommendations, the business can leverage the model to enhance productivity, reduce manual labor, and make more informed decisions in agriculture, leading to better crop management and higher yields.
from tensorflow.keras.preprocessing.image import ImageDataGenerator
# Creating the data generator with multiple augmentations
train_datagen = ImageDataGenerator(
rotation_range=20,
width_shift_range=0.2,
height_shift_range=0.2,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True,
#brightness_range=[0.8, 1.2],
fill_mode='nearest'
)
# Take a random sample from the training data to visualize augmentation
sample_index = np.random.randint(0, len(X_train_normalized))
sample_image = X_train_normalized[sample_index]
sample_label = y_train_encoded[sample_index]
# Ensure sample_label is a numpy array
sample_label = np.array(sample_label)
# Generate batches of augmented images
augmented_images = train_datagen.flow(
sample_image.reshape(1, 64, 64, 3),
batch_size=1
)
# Plotting original and augmented images
plt.figure(figsize=(12, 6))
# Original image
plt.subplot(1, 5, 1)
plt.imshow(sample_image)
plt.title(f'Original\nLabel: {enc.inverse_transform(sample_label.reshape(1, -1))[0]}')
plt.axis('off')
# Augmented images
for i in range(4):
aug_img = augmented_images.next()[0]
plt.subplot(1, 5, i + 2)
plt.imshow(aug_img)
plt.title(f'Augmented\nLabel: {enc.inverse_transform(sample_label.reshape(1, -1))[0]}')
plt.axis('off')
plt.tight_layout()
plt.show()